Way Memoization to Reduce Fetch Energy in Instruction Caches

Instruction caches consume a large fraction of the total power in modern low-power microprocessors. In particular, set-associative caches, which are preferred because of lower miss rates, require greater access energy on hits than direct-mapped caches; this is because of the need to locate instructions in one of several ways. Way prediction has been proposed to reduce power dissipation in conventional set-associative caches; however, its application to CAMtagged caches, which are commonly used in low-power designs, is problematic and has not been quantitatively examined. We propose way memoization as an alternative to way prediction. As in way prediction schemes, way memoization stores way information (links) within the instruction cache, but in addition maintains a valid bit per link that when set guarantees that the way link is valid. In contrast, way prediction schemes must always read one tag to verify that the prediction is correct. Way memoization requires a link invalidation mechanism to maintain the coherence of link information. We investigate several invalidation schemes and show that simple conservative global invalidation schemes perform similarly to exact invalidation schemes but with much lower implementation cost. Based on HSPICE simulations of complete cache layouts in a 0.25 m CMOS process, and processor simulations running SPECint95 and MediaBench benchmarks, we show that way memoization reduces the energy of a highly-optimized 16 KB 64-way CAM-tag low-power instruction cache by 21%, an additional 13% savings compared to a way-predicting cache.